Tactile-Vision Integration for Task-Compatible Fine-Part Manipulation

We propose to integrate tactile and visual sensing to predict task-compatible grasp regions for manipulation. We address the problem of fine-part assembly, by leveraging vision to observe scene-level information, then touch to perceive fine local details necessary for task completion. We directly target the end goal of task success, by predicting grasp regions that are simultaneously geometrically stable and task-compatible. We represent grasp regions with 2D probabilistic maps, which we first coarsely estimate with vision, and then refine by making contacts with the scene. We show preliminary results of probabilistic grasp regions generated by vision, and demonstrate the impact of tactile sensors for disambiguating task-compatible regions.